1. Field of the Invention
The present invention generally relates to a method and an apparatus for image signal encoding, and particularly relates to a method and an apparatus for image signal encoding, which encodes image signals at variable bit rates.
2. Description of the Related Art
Recently, such an international standard as MPEG (Moving Picture Experts Group) is employed in an image signal encoding apparatus. In such an apparatus, a VBV (Video Buffering Verifier) is virtually connected to a bitstream output point of the image signal encoding apparatus so that the bitstream shall not collapse a receiving buffer of the image signal decoding apparatus by preventing a collapse at the VBV.
FIG. 1 shows an example of VBV buffer occupancy index that varies with time. At a time A, a bitstream starts to flow into the VBV, and the occupancy increases as time elapses. At a time B, the occupancy decreases by data volume D which is the volume of the data of an image frame then decoded. From the time B to a time C, the buffer occupancy index increases. At the time C, the occupancy decreases by data volume E that is the volume of the data of an image frame then decoded. In FIG. 1, a slope of the buffer occupancy while the buffer occupancy index increases represents a transmission rate of the bitstream supplied to the VBV.
In the image signal encoding apparatus, it is necessary to control the encoding so that the buffer occupancy is between 0 and a given buffer size. If the control is not appropriate, then the VBV will collapse as shown in FIGS. 2 and 3.
FIG. 2 shows an example of an underflow that occurs at the VBV. At a time F, a bitstream starts flowing into the VBV, increasing the buffer occupancy index. At a time G, the occupancy decreases by data volume I which is the volume of the data of an image frame then decoded. The buffer occupancy index increases from the time G to a time H.
At the time H, not all of the bitstream for an image frame to be decoded has arrived. The image frame cannot be decoded. This situation is called a buffer underflow.
Conversely, FIG. 3 shows an example of an overflow. The bit rate control method in a constant rate encoding mode is different from that in a variable bit rate encoding mode. The overflow of the VBV occurs in the constant bit rate encoding mode.
At a time K, a bitstream starts flowing into the VBV to increase the occupancy. At a time L, an image frame with data volume N is decoded, and decreases the buffer occupancy index by N. From the time L to a time M, the buffer occupancy index increases.
At the time M, an image frame is to be decoded. However, volume of data that was decoded at the time L was small, which causes the buffer occupancy index of the VBV to reach the given buffer size before the time M. For this reason, at the time M, not all of necessary bitstream has been received and a normal decoding is impossible. This situation is called an overflow.
FIG. 4 shows an example of the buffer occupancy index behavior with time in the variable bit rate encoding mode. In the variable encoding mode, there is no buffer overflow. When the buffer occupancy index reaches at the given buffer size, then the bitstream stops flowing. As the buffer occupancy index decreases, the bitstream resumes flowing.
At a time P, a bitstream starts flowing into the VBV, increasing the buffer occupancy index. At a time Q, an image frame of which data size is T is decoded, thereby decreasing the occupancy index by T. From the time Q to a time R, the buffer occupancy index increases.
At the time R, the buffer occupancy index reaches at the given buffer size, thereby stopping the bitstream flowing to the VBV. At a time S, an image frame of which data volume is U is decoded, which makes the buffer occupancy index decrease by U. Then, the bitstream to the VBV resumes flowing.
The overflow and underflow of the buffer occur where the originated data volume control at encoding is inappropriate. A proper control of the originated data volume at encoding is necessary. In order to control the originated data volume at encoding, an appropriate target data volume is set to an image frame. The originated data volume is controlled so as to meet the target volume.
Conventionally, such a control as follows has been practiced in order to make the originated data volume to approximate the target volume. In the following description of an example, T represents the target data volume of an image frame, d0 represents an initial occupancy of a quantization control buffer, dj represents an occupancy of the quantization control buffer immediately before encoding an encoding block (macroblock) j, Bj represents an originated data volume from a first encoding block (macroblock) of the image frame to the j-th encoding block (macroblock), MBcnt represents a number of encoding blocks (macroblocks) in an image frame, Q represents a quantization scale code by which the j-th encoding block (macroblock) is quantized and r represents an reaction parameter.
The reaction parameter r is given by a formula (1) presented below by a frame rate of a moving image xe2x80x9cpicture_ratexe2x80x9d and an encoding bit rate xe2x80x9cbitratexe2x80x9d.
r=2xc3x97(bitrate)/(picture_rate)xe2x80x83xe2x80x83(1)
An encoding block (macroblock) is made of a plurality of pixels. As shown in FIG. 5, the occupancy of the quantization control buffer when encoding the j-th encoding block (macroblock) is given by a formula (2) below. In FIG. 5, the hatched blocks represent encoded macroblocks and others represent macroblocks that have not been encoded.
dj=d0+Bjxe2x88x921xe2x88x92(Txc3x97(jxe2x88x921))/MBcntxe2x80x83xe2x80x83(2)
Thus calculated quantization control buffer occupancy index is substituted to a formula (3) below, to obtain a quantization scale code by which the j-th macroblock is to be quantized.
Q=djxc3x9731/rxe2x80x83xe2x80x83(3)
The scale code Q thus calculated by the formula (3) is substituted to a formula (4) below to obtain the quantization scale QS.
QS=Qxc3x972xe2x80x83xe2x80x83(4)
By performing the process described above for all the macroblocks in the image frame, the originated data volume approximates to the target data volume. If the originated data volume is greater than the target data volume, then the quantization scale QS is enlarged to reduce the originated data volume, and if the originated data volume is smaller than the target data volume, then the quantization scale QS is made smaller to increase the originated data volume. In this manner, the receiving buffer collapse has been suppressed.
In a conventional image signal encoding apparatus, the constant bit rate encoding mode has been employed, which transfers encoded image signal at a constant encoding bit rate due to requirements from networks or the like. In the constant bit rate encoding mode, however, there is a tendency to build picture quality differences between images that are difficult to compress because of complexity and abrupt motions in the image and relatively still images that are easy to compress. On the other hand, in the variable bit rate encoding mode, a higher encoding bit rate is used for images that are difficult to compress and a lower encoding bit rate is used for images that are easy to compress so that there is no quality difference between the hard-to-compress images and the easy-to-compress images.
With recent availability of wideband asynchronous networks such as IP (Internet Protocol) networks and developments of storage media compliant with the variable bit rate encoding mode, the variable bit rate encoding mode for compressed image signals has been developed and put into practice.
A variable bit rate encoding mode used in DVD (Digital Versatile Disk) is called an n-path variable bit rate encoding method and requires two or more encoding steps to compress-encode moving pictures. For example, 2-path variable bit rate encoding method first determines a degree of difficulty in compression of each compressed image frame by encoding the frame at a constant compression ratio. Then, in a second encoding, each frame is encoded based upon data volume of each image frame. Therefore, the n-path variable bit rate encoding method cannot perform a realtime encoding of image frames.
In a disk recording apparatus and a disk installed camcorder that use a hard disk, DVD-RAM (DVD-Random Access Memory) and the like, the variable bit rate encoding mode is employed to enable recording of a high quality motion picture for a number of hours. A realtime encoding of image frames is necessary. To realize the realtime encoding, a method has been developed in which a degree of the difficulty in encoding a next frame is predicted from statistical information of frames already encoded, and based on which and observing an average encoding bit rate, the realtime image frame encoding is performed.
Further, in the variable bit rate encoding mode that uses the statistical information of encoded frames, a target data volume for an image frame is set, and the encoding is performed to originate data volume to an approximation of the target volume, similarly to the constant bit rate encoding mode. However, in an encoding bit rate equal to or less than 2 Mbps (bits per second), the size of an image frame is a bigger than 720xc3x97480, and a wrong prediction of originated data volume for an image frame creates a following problem.
For example, if a compression of a next image frame is predicted easy and set the target originating data volume small, however, actual image is found to be difficult to compress, then picture quality is deteriorated due to a high compression ratio to compress the image to the small target originated data volume.
Accordingly, it is an object of the present invention to provide a method and an apparatus for image encoding which enable high picture quality variable bit rate encoding that is responsive to abrupt changes of the images.
The above object of the present invention is achieved by an image signal encoding method in which an image frame is quantized by a quantized scale and thus quantized image frame is encoded includes a step wherein a predicted data volume is calculated for a given encoding block (macroblock) of a target encoding image frame from a calculated maximum data volume after the maximum data volume and a first quantization scale are calculated based on an encoding result of previously encoded image frames, a step in which the encoding block (macroblock) of the encoding target image frame is quantized by the quantization scale and an originated data volume is calculated for an encoded image frame, a step in which the predicted data volume is compared with the originated data volume, result of which is used to calculate a second quantization scale for updating and a step in which the first quantization scale is compared with the second quantization scale, the larger of the two to be used for updating as a new quantization scale.
In an image signal encoding method as above, the first quantization scale based on an encoding result of image frames previously encoded is compared with the second quantization scale which is based on a comparison of the maximum data volume and the originated data volume, and the larger of the two is taken as a new quantization scale. The second quantization scale respects the maximum data volume, thereby enabling to prevent a collapse of a quantization control buffer and a violation of a maximum encoding bit rate when the quantization is performed at a quantization scale that is at least greater than the second quantization scale.
The present invention which is also related to an image signal encoding apparatus in which an image frame is quantized by a quantization scale and thus quantized image frame is then encoded includes a first quantization scalexe2x80x94maximum data volume calculation unit, a quantization control buffer calculation unit, and a quantization control unit. The first quantization scalexe2x80x94maximum data volume calculation unit calculates the maximum data volume and the first quantization scale based on an encoding result of image frames previously encoded. The quantization control buffer calculation unit calculates a predicted data volume for a given encoding block (macroblock) in the target encoding image frame from the maximum data volume, quantize the macroblock that belongs to the target encoding image frame by the quantization scale, and calculates the second quantization scale based on a difference between the predicted data volume and the originated data volume that is the volume of the data originated when the quantized image frame is encoded. The quantization control unit compares the first quantization scale with the second quantization scale and outputs the larger of the two as the new quantization scale.
The image signal encoding apparatus as above includes the first quantization scalexe2x80x94maximum data volume calculation unit that calculates the first quantization scale based on the encoding result of image frames already encoded, the quantization buffer calculation unit that calculates the second quantization scale based on a comparison of the maximum data volume with the originated data volume and the quantization control unit that compares the first quantization scale with the second quantization scale to output the larger of the two scales, thereby performing a quantization at a quantization scale at least the second quantization scale that respects the maximum data volume thus enabling to prevent a collapse of the quantization control buffer and a violation of the maximum encoding bit rate.
Other objects and further features of the present invention will be apparent from the following detailed description when read in conjunction with the accompanying drawings.